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Abstract: We review results on linearly edge-reinforced random walks. On 
finite graphs, the process has the same distribution as a mixture of reversible 
Markov chains. This has applications in Bayesian statistics and it has been 
used in studying the random walk on infinite graphs. On trees, one has a 
representation as a random walk in an independent random environment. We 
review recent results for the random walk on ladders: recurrence, a representa- 
tion as a random walk in a random environment, and estimates for the position 
of the random walker. 



1. Introduction 

Consider a locally finite graph G = (V, E) with undirected edges. All the edges 
are assigned weights which change in time. At each discrete time step, the edge- 
reinforced random walker jumps to a neighboring vertex with probability propor- 
tional to the weight of the traversed edge. As soon as an edge is traversed, its weight 
is increased by one. 

The model was introduced by Diaconis in 1986 (see and Q). Mike Keane 
made the model popular in The Netherlands. In particular, Mike Keane introduced 
Silke Rolles to the intriguing questions raised by this model. One fundamental 
question, asked by Diaconis, concerns recurrence: Do almost all paths of the edge- 
reinforced random walk visit all vertices infinitely often? By a Borel-Cantelli argu- 
ment, this is equivalent to the following question: Does the edge-reinforced random 
walker return to the starting point infinitely often with probability one? For all 
dimensions d > 2, it is an open problem to prove or disprove recurrence on Z d . 
Only recently, recurrence of the edge-reinforced random walk on ladders has been 
proven. 

The present article focuses on linear edge-reinforcement as described above. In 
the past two decades, many different reinforcement schemes have been studied. We 
briefly mention some of them: Vertex-reinforced random walk was introduced by 
Pemantle [IE] and has been further analyzed by Benai'm Q, Dai (0, 01 Peman- 
tle and Volkov HI), and Tarres O. Sellke [Hjj and VervoortTnt proved 
recurrence results for once-reinforced random walks on ladders; Durrett, Kesten, 
and Limic 0] analyzed the process on regular trees. Recurrence questions for re- 
inforced random walks of sequence type were studied by Davis (0, @) and Sellke 
|3lj . Takeshima 13) studied hitting times and recurrence for reinforced ran- 
dom walk of matrix type. Limic [lij proved a localization theorem for superlinear 
reinforcement. Weakly reinforced random walks in one dimension were studied by 
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Toth [36j. Directionally reinforced random walks are the subject of Mauldin, Monti- 
cino, and von Weizsacker [2cj and Horvath and Shao ^4|. Othmer and Stevens [23| 
suggested linearly edge-reinforced random walks as a simple model for the gliding 
of myxobacteria. These bacteria produce a slime and prefer to move on the slime 
trail produced earlier. Overviews with a different emphasis than the present article 
have been written by Davis and Pemantle [2(|. 

The article is organized as follows: In Section |2 we give a formal definition 
of the edge-reinforced random walk. On finite graphs, the edge-reinforced random 
walk has the same distribution as a random walk in a dependent random envi- 
ronment. This representation together with related limit theorems is presented in 
Section 13 The edge-reinforced random walk on finite graphs gives rise to a family 
of prior distributions for reversible Markov chains. This application to statistics is 
the content of Section A characterization of the process (which is also applied 
in the Bayesian context) is given in Section In Section El we state results for 
the process on acyclic graphs. Recently, progress has been made in understanding 
the edge- reinforced random walk on graphs of the form Z x {1, . . . , d} and, more 
generally, onZxT when T is a finite tree. Section [7| reviews these recent results 
and presents some simulations. 

2. Formal description of the model 

Let G = (V, E) be a locally finite undirected graph. Every edge is assumed to have 
two different endpoints; thus there are no direct loops. We identify an edge with 
the set of its endpoints. Formally, the edge-reinforced random walk on G is defined 
as follows: Let X t denote the random location of the random walker at time t. Let 
a e > 0, e € E. For t € No, we define w t (e), the weight of edge e at time t, recursively 
as follows: 



Let P VQ ,a denote the distribution of the edge-reinforced random walk on G start- 
ing in vq with initial edge weights equal to a — (a e ) ee E- The distribution P Vo>a is a 
probability measure on V N ", specified by the following requirements: 



wo(e) :=a e for all e € E, 



(2.1) 




fore={X t ,X t+1 }eE, 
for eeE\{{X t ,X t+1 }}. 



(2.2) 



Xq = Vq P VOja -&.S., 



(2.3) 



w t ({X u v}) 



if {X u v}eE, 



P Vo , a (X t+1 =v\Xi,i = 0,l,...,t) 



T,{e€E:X t ee} W «( e ) 




otherwise. 



(2.4) 



3. Reinforced random walk on finite graphs 



Throughout this section, we assume the graph G to be finite. 
For t e N and eE E, set 

hi 

k t (e) := w t (e) - a e and a t (e) := — 



(3.1) 
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In particular, a t (e) denotes the proportion of crossings of the edge e up to time t. 
The random vector a t := (a t (e)) e£ E takes values in the simplex 



A := | (x e ) eeE G (0, ^ f : ^ x e = 1 \ . (3.2) 
For x G (0, oo)^ and v G V, we define 



eS-E 



(3.3) 



Now view the graph G as an electric network, and consider the space Hi of all 
electric current distributions (y e ) e eE on the graph, such that Kirchhoff's vertex 
rule holds at all vertices: For every vertex, the sum of all ingoing currents should 
equal the sum of all outgoing currents. More formally, we proceed as follows: We 
assign a counting direction to every edge, encoded in a signed incidence matrix 
s = {s ve )vev,eeE G { — 1, 0, l} VxE . Here, s ve = +1 means that e is an ingoing edge 
into the vertex v; s ve = —1 means that e is an outgoing edge from vertex v, and 
s ve = means that the vertex v is not incident to the edge e. Then, for a given 
current distribution y = (y e )eeE € R E , the current balance at any vertex v e V 
equals 

e€E 

and the space of all current distributions satisfying Kirchhoff's vertex rule equals 
Hi = kernel(s) = {y e R E : sy = 0} 

- I (y e )eeE G R E : s veVe = for all v G V \ . (3.5) 

I ee-E J 

In other words, -f/i is the first homology space of the graph G. Note that dimiJi = 
\E\ -\V\ + l by Euler's rule. 

Interpreting weights x = (x e ) eg £ G (0,oo) E as electric conductivities of the 
edges, the power consumption of the network required to support a current distri- 
bution y G Hi equals 

My) = Y y f- 

eEE e 

(Of course, in order to drive this current in a real network, appropriate batteries 
must be wired into the edges. We ignore this fact.) Note that A x is a quadratic form 
on Hi. We will also need its determinant det A x , which is defined as the determinant 
of the matrix representing A x with respect to any Z-basis of the lattice Hi (1 Z E . 
Note that the determinant det A x does not depend on the choice of the basis, since 
any base change has determinant ±1. We endow the homology space Hi with the 
Lebesgue measure, normalized such that the unit cell spanned by any Z-basis of 
Hi n Z B gets volume 1. 

It turns out that the sequence (at)ten converges almost surely to a random limit. 
Surprisingly, the limiting distribution can be determined explicitly: 

Theorem 3.1. The sequence (a t )teN converges P Vo ^ a -almost surely. The distribu- 
tion of the limit is absolutely continuous with respect to the surface measure on A 



Linearly edge- reinforced random walks 



6f) 



with density given by 

n 

<Pv A x ) = Z vo,a ^o. &eE ^rr\/det A x , x = (x e ) eeE £ A, (3.6) 

vev\{v ] 

where Z Vo ^ a denotes a normalizing constant. 

Theorem 13.11 was discovered by Coppersmith and Diaconis Q . A proof of the 
result was independently found by Keane and Rolles ([I3> Theorem 1). In the 
special case of a triangle, a derivation of Theorem l3.1l was published by Keane [lij . 
The normalizing constant Z Vo ^ a is explicitly known; see |l6j. 

Let S denote the set of all subtrees T in G, viewed as set of edges T C E, 
which need not necessarily visit all vertices. The set of all spanning trees in G is 
denoted by T; it is a subset of S. Using a matrix tree theorem (see e.g. ,19], page 
145, theorem 3'), one can rewrite the determinant of A x as a sum over all spanning 
trees. This yields the following representation of the density: 

UK'- 1 , 

4>v 0>a ( x ) = C"s^ — ^+ijy] n Xe - ( 3 - ? ) 

Xvo II X v 2 \j TeTeGT 

v£V\{v } 

Given a path n = (vq,Vi, . . . ,v t ) of vertices in G, we define its corresponding 
"chain" [it] = ([ir] e ) e£ E & R B as follows: Imagine an electric current of size 1 
entering the network G at the starting vertex vq, flowing along tt, and then leaving 
the network at the last vertex v t . Let [tt] £ R s denote the corresponding current 
distribution. More formally, for e £ E, [ir] e equals the number of times the path 
7r traverses the edge e in counting direction minus the number of times the path 
7r traverses the edge e in opposite direction to the counting direction. Note that 
[it] ^ Hi unless the path it is closed, since Kirchhoff 's vertex rule is violated at the 
starting point vq and at the last vertex Vt- Now, for any time t, let 

T^i = (Xq,Xi, . . . ,X t ) 

denote the random path the reinforced random walker follows up to time t. We 
define 

0t= ^Nei E (3.8) 

to be the corresponding rescaled (random) current distribution. Indeed, the diffusive 
scale y/i turns out to be appropriate for studying the random currents [irt] in the 
limit as t — > oo. Note that in general fit 4- due to the violation of Kirchhoff 's 
rule at the starting vertex Xq and the last vertex Xt. However, any weak limit of (3t 
as t — > oo, if it exists, must be supported on the homology space Hi C R E . Indeed, 
the violation of Kirchhoff 's rule at any vertex is not larger than £~ 1//2 , which is 
negligible as t — > oo. 

For a (random) path irt = {Xq, . . . , Xt) and any vertex v among {Xq, . . . , Xt-i\, 
let e last oxlt (w, Tit) denote the edge by which v is left when it is visited by irt for the 
last time. Let T t last exlt £ S denote the random tree graph consisting of all the edges 
e CXIt (u, 7r t ), v £ {Xq, . . . , X t -i}. Since the reinforced random walk on the finite 
graph G visits every vertex almost surely, T/ ast CXIt £ T holds for all large t almost 
surely. 
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Theorem 3.2. The sequence (a t , /3 t , T t last cxlt ) te N converges weakly in R E x R B x S. 
The limiting distribution Q is supported on the subset A x Hi x T ofM. E x R B x S . 
It is absolutely continuous with respect to the product of the surface measure on A, 
the Lebesgue measure on Hi, and the counting measure on T . Its density is given 
by 



$ VOta (x,y,T) = Z-) a ^^ ^ ( J] x e ) exp (-U x (y)) , (3.9) 

XvS II X' v 2 VeGT / V & J 

v£V\{v } 



where Z VQ ^ a denotes a normalizing constant. 

Theorem 1 of 0| states weak convergence of (cxt,Pt)teN to the limiting distri- 
bution with density 



E 



$ Vo , a (x,y,T). (3.10) 



However, the proof implicitly contains the proof of Theorem 13 .21 

Recall that the transition probabilities of any irreducible reversible Markov chain 
on G can be described by weights x = {x e ) ee E, x e > 0, on the edges of the graph; 
the probability to traverse an edge is proportional to its weight. More precisely, 
denoting the distribution of the Markov chain induced by the edge weights x with 
starting vertex vq by Q Vo ,x, one has 

Q Vo>x (X t+ i = v'\X t = v) = (3.11) 

x v 

whenever {v,v'} is an edge; the weight x v is defined in (|3.3I) . 

The following representation of the edge-reinforced random walk on a finite graph 
as a mixture of reversible Markov chains is shown in Theorem 3.1 of |2*sj . 

Theorem 3.3. For any event B C V N ° , one has 



P„ ,„((I f ) teNo £B)= / Q VOtX (B)Q(dxdydT), (3.12) 

AxffixT 

where Q denotes the limiting measure from Theorem \3.2L In other words, the edge- 
reinforced random walk on any finite graph G has the same distribution as a random 
walk in a random environment. The latter is given by random weights on the edges, 
distributed according to the limiting distribution of (cxt)teN> namely the distribution 
with density <fi Vo , a with respect to the Lebesgue measure on the simplex. 

This representation as a random walk in a random environment has been ex- 
tremly useful in studying the edge-reinforced random walk on infinite ladders. 

The proof of Thcorcm l3.3l rclics on the fact that the edge-reinforced random walk 
is partially exchangeable: the probability that the process traverses a particular fi- 
nite path 7r = (vq, . . . ,vt) depends only on the starting point of ir and the number 
of transition counts of the undirected edges. If one knows that the process returns 
to the starting point infinitely often with probability one (which is the case for the 
edge- reinforced random walk on a finite graph), one can apply a de Finetti Theo- 
rem for Markov chains of Diaconis and Freedman . A refinement for reversible 
Markov chains (|28j. Theorem 1.1) yields a mixture of reversible chains. 
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4. An application to Bayesian statistics 

Consider the following statistical situation: We observe X = Vq, X\ = V\, . . ., 
Xt = Vt generated by a reversible Markov chain. The transition kernel fc(v) an( i 
the stationary measure v are unknown. Let V be the set of possible observations. 
We assume V to be a known finite set. Furthermore, we assume that k(v,v') > if 
and only if k(v' , v) > 0. Hence, V together with the set E := {{v, v'} : k(v, v') > 0} 
defines a finite undirected graph. This graph is assumed to be known. 

It is a natural question how to model this in the framework of Bayesian statistics. 
One is interested in "natural" prior distributions on the set of reversible Markov 
chains. 

Because of reversibility, u(v)k(v,v') — v(v')k{v' , v) for all v,v' <E V. The distri- 
bution of a reversible Markov chain with transition kernel k, stationary distribution 
u, and starting point vq is given by Q VOtX as defined in (|3 - 1 1|> with edge weights 
x {v,v'} = v{v)k(v,v'). Thus, one can describe the prior distributions as measures 
on the set of possible edge weights, namely as measures on A. 

The following theorem was proved by Diaconis and Rolles (Proposition 4.1 of 
0): 

Theorem 4.1. Let a denote the Lebesgue measure on A and set P^ , a := <Pv ,ado~. 
The family 

(P, ,a : wo€V r ,«= (a e )eeE € (0,oo) E } (4.1) 

of prior distributions is closed under sampling. More precisely, under the prior 
distribution P„ 0)O with observations Xq = vq, X\ — v\, . . ., X t — v t , the posterior 
is given by ^"v t ,(a c +k t (e))^ eE > where k t (e) denotes the number ofi with {vi, = e. 

These prior distributions were further analyzed in [l^ |: They can be generalized 
to finite graphs with direct loops; thus one can include the case k(v,v) > for 
some v. The set of linear combinations of the priors P« 0l a is weak-star dense in the 
set of all prior distributions on reversible Markov chains on G. Furthermore, it is 
shown that these prior distributions allow to perform tests: several hypotheses are 
tested for a data set of length 3370 arising from the DNA sequence of the humane 
HLA-B gene, for instance Hq : i.i.d. (unknown) versus Hi : reversible Markov chain 
and Hq : reversible Markov chain versus Hi : full Markov. The tests are based on 
the Bayes factor P(data|iJo)/-P(data|iJi) which can be easily computed. 

The priors P^ 05 a generalize the well-known Dirichlet priors. The latter are ob- 
tained as a special case for star-shaped graphs. The Dirichlet prior was characterized 
by W.E. Johnson; see In ^3], a similar characterization is given for the priors 
Pti , a ; in this sense they are "natural". 

5. A characterization of reinforced random walk 

In this section, we review a characterization of the edge-reinforced random walk 
from 5s| . We need some assumptions on the underlying graph G: 

Assumption 5.1. For all v G V, degree(u) ^ 2. Furthermore, the graph G is 
2-edge-connected, i.e. removing an edge does not make G disconnected. 

Let P be the distribution of a nearest-neighbor random walk on G such that the 
following hold: 

Assumption 5.2. There exists vq S V with P(Xq = Vq) = 1. 
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Assumption 5.3. For any admissible path 7r of length t > 1 starting at Vq, we 
haveP((X ,...,Xt) = 7r)>0. 

Assumption 5.4. The process (X t )teN with distribution P is partially exchange- 
able. 

For t £ Nq, v £ V, and e G E, we define 

k t (v) := |{t G {0, . . . ,*} : Xi = v}\, k t {e) := |{t G {1, .. . ,t} : {X^,X t } = e}\. 

Assumption 5.5. For all v E V and e E E, there exists a function f ve taking 
values in [0, 1] such that for all f £ No 

P(X t+1 = v\X , ...,X t ) = f Xt< {x t ,v}( k t( X t)> kt({X t ,v})). 

In other words, the conditional distribution for the next step, given the past up to 
time t, depends only on the position X t at time t, the edge {X t , v} to be traversed, 
the local time accumulated at X t , and the local time on the edge {X t ,v}. 

It is not hard to see that an edge-reinforced random walk and a non-reinforced 
random walk starting at vq satisfy Assumptions I5.2H5.5I The following theorem is 
the content of Theorem 1.2 of pif: 

Theorem 5.1. Suppose the graph G satisfies Assumvtion \5.1\ If P is the distribu- 
tion of a nearest-neighbor random walk on G satisfying Assumvtions \5.^l5.5l then, 
for all t, 

P(X t+1 =v\X ,...,X t ) 

agrees on the set {k t (X t ) > 3} with the corresponding conditional probability for an 
edge-reinforced random walk or a non-reinforced random walk starting at Vq- 

In this sense, the above assumptions characterize the edge-reinforced random 
walk. Theorem 15 .11 is used to give a charactarization of the priors P Vo .a- 

6. Reinforced random walk on acyclic graphs 

The edge-reinforced random walk on acyclic graphs is much easier to analyze than 
on graphs with cycles. Let us briefly explain why: Consider edge-reinforced random 
walk on a tree. If the random walker leaves vertex v via the neighboring vertex v 1 , 
then in case the random walker ever returns to v, the next time she does so, she 
has to enter via the same edge {v, v'}. Hence, if the random walker leaves v via the 
edge {«, v'y, the weight of this edge will have increased by precisely 2, the next time 
the random walker arrives at v. Obviously, this is only true on an acyclic graph. 

Due to this observation, instead of recording the edge weights, one can place 
Polya urns at the vertices of the tree. Each time the random walker is at location v, 
a ball is drawn from the urn U{v) attached to v. Then, the ball is put back together 
with two balls of the same color. The different colors represent the edges incident 
to v; the number of balls in the urn equal the weights of the edges incident to v, 
observed at times when the random walker is at v. The initial composition of the 
urns is determined by the starting point and the initial edge weights of the reinforced 
random walk. The sequence of drawings from the urn U(v) at v is independent from 
the sequences of drawings from the urns U(v'), v ^ v', at all other locations. Using 
de Finetti's theorem, one finds that the edge-reinforced random walk has the same 
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distribution as a random walk in a random environment, where the latter is given by 
independent Dirichlet-distributed transition probabilities. This representation as a 
random walk in a random environment was observed by Pemantle |24j | . It seems to 
be the most powerful tool to analyze the process on acyclic graphs. Pemantle used 
it to prove a phase transition in the recurrence/transience behavior on a binary 
tree. Later, Takeshima |33| characterized recurrence in one dimension for space- 
inhomogeneous initial weights. (For all initial weights being equal, recurrence in one 
dimension follows for instance from a well-known recurrence criterion for random 
walk in random environment; see e.g. |32j.) Collevecchio 3 proved a law of large 
numbers and a central limit theorem for the edge-reinforced random walk on 6-ary 
trees, under the assumption that b is large. 

For linear edge-reinforcement on arbitrary directed graphs, so called directed- 
edge-reinforced random walk, a similar correspondance with a random walk in an 
independent random environment can be established. Using this correspondance, 
recurrence for Z x G for any finite graph G was proved by Keane and Rollcs in 
|l7| . A different criterion for recurrence and transience of a random walk in an 
independent random environment on a strip was established by Bolthausen and 
Goldsheid 0. 

7. Reinforced random walk on ladders 

Studying reinforced random walks on graphs of the form Z x {1, . . . , d} seems to be 
a challenging task. Only recently, recurrence results were obtained. The following 
result was proved in [22j. 

Theorem 7.1. For all a > 3/4, the edge- reinforced random walk on Z x {1,2} with 
all initial weights equal to a is recurrent. 

The result was extended in |29| . under the assumption the initial weight a is 
large: 

Theorem 7.2. Let G be a finite tree. For all large enough a, the edge-reinforced 
random walk on Z x G with all initial weights equal to a is recurrent. 

In particular, this applies to ladders Z x {1, 2, . . . , d} of any finite width d. 

In the following, we consider the process on No x G or Z x G with a finite tree G. 
We always assume that all initial edge weights are equal to the same large enough 
constant a and the random walk starts in a vertex at level 0. 

Just as edge-reinforced random walk on finite graphs, the edge-reinforced ran- 
dom walk on infinite ladders turns out to be equivalent to a random walk in a 
random environment, as studied in Section [3J In particular, there is an infinite- 
volume analogue to Theorem 13.31 for the infinite graphs No x G and Z x G. Here, 
the law Q of the random environment in Theorem 13.31 gets replaced by an infinite- 
volume Gibbs measure, which we also denote by Q. Also in the infinite- volume 
setup, the fractions at £ M. E of times spent on the edges converge almost surely to 
random weights x € M. E as t — > oo, just as in Theorem 13. II These random weights 
x are governed by the infinite- volume Gibbs measure Q. It turns out that Q-almost 
surely, the random weights decrease exponentially in space, i.e. 

Q-a.s. lim sup ^—r log x e < — c(a, G) (7.1) 

|e|-«50 |e| 

with a deterministic constant c(G, a) > 0. Here, lei denotes the distance of an edge 
e from the starting point. For more details, see )2l\ . 
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1e-8 1e-7 1e-6 1e-5 1e-4 1e-3 1e-2 

Fig 1. Fraction of time spent on the edges on a logarithmic scale. Time horizon= 10 9 , initial 
weight a = 1. 



This exponential decay of the weights x can be also observed in simulations: For 
the ladder Z x {1, 2, . . . , 30}, the simulation in Figuren snows one typical sample 
for the fraction of time a t spent on any edge up to time t = 10 9 . The fractions 
a t are displayed logarithmically as gray scales. The starting point is located in the 
center of the picture. The time t = 10 9 in the simulation is already so large that 
this at is a good approximation for a typical sample of the random weights x. 

Note that the fractions at in the simulation vary over many orders of magnitude, 
and that they decrease roughly exponentially as one gets farther away from the 
starting point. 

Entropy estimates and deformation arguments from statistical mechanics are 
used in [21j to derive the exponential decay of the weights. 

As a consequence, one obtains the following estimates for the position of the 
random walker, also proved in [2l|: 

Theorem 7.3. There exist constants c\,C2 > 0, depending only on G and a, such 
that for all t,n gNq, the following bound holds: 

PoA\Xt\>n)<c ie ~ C2n . (7.2) 

Note that the bounds are uniform in the time t. This is different from the behavior 
of simple random walk, which has fluctuations of order \fi. 

Corollary 7.1. There exists a constant C3 = c^(G, a) > such that Po ta -a.s., 

max \X S \ < C3lni for all t large enough. (7-3) 

s=0,...,t 

A simulation shown in Figure El illustrates this corollary: For one sample path of 
an edge-reinforced random walk on the ladder Z x {1, 2, . . . , 30} with initial weight 
a = 1, the farthest level reached so far, max s= o ) ... i t \X S \, is displayed as a function 
of t. Note that the time t is plotted on a logarithmic scale. 

Figure |3 shows a simulation of reinforced random walk on 1? with initial weights 
a = 1. It is still unknown whether this reinforced random walk is recurrent. Simu- 
lations show that there are random regions in Z 2 , maybe in far distance from the 
starting point, which are visited much more frequently than other regions closer to 
the starting point. It remains unclear whether more and more extreme "favorable 
regions" arise farther away from the origin. Thus, the recurrence problem in 1? 
remains open, even on a heuristic level. 
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1 ' => ■ ' ' 1 — ' ^ — ' ^ — ' ^ — ' ^ — ' ^ — ' ^ — ' ^ 

1 10 100 1000 10000 100000 1e+06 1e+07 1e+08 1e+09 

time t 

Fig 2. Maximal distance from the starting point as a function of time. Initial weight a = 1. 




Fig 3 . Reinforced random walk on I? . 



76 



F. Merkl and S. W. W. Rolles 



References 



Ben aim, M. (1997). Vertex-reinforced random walks and a conjecture of Pc- 
mantle. Ann. Probab. 25, 1, 361-392. MR1428513 

Bolthausen, E. and Goldsheid, I. (2000). Recurrence and transience of 
random walks in random environments on a strip. Comm. Math. Phys. 214, 
no. 2, 429-447. MR1796029 

Coppersmith, D. and Diaconis, P. (1986) Random walk with reinforce- 
ment. Unpublished manuscript. 

Collevecchio, A. (2005). Limit theorems for Diaconis walk on certain trees. 
Preprint. 

Dai, J. J. (2003). A note on vertex-reinforced random walks. Statist. Probab. 
Lett. 62, 3, 275-280. MR1966379 

Dai, J. J. (2004). Some results regarding vertex-reinforced random walks. 
Statist. Probab. Lett. 66, 3, 259-266. MR2045471 

Davis, B. (1989). Loss of recurrence in reinforced random walk. In Almost 
everywhere convergence (Columbus, OH, 1988). Academic Press, Boston, MA, 
179-188. MR1035245 

Davis, B. (1990). Reinforced random walk. Probab. Theory Related 
Fields 84, 2, 203-229. MR1030727 

Davis, B. (1999). Reinforced and perturbed random walks. In Random walks 
(Budapest, 1998). Bolyai Soc. Math. Stud., Vol. 9. Janos Bolyai Math. Soc, 
Budapest, 113-126. MR1752892 

DIACONIS, P. and Freedman, D. (1980). de Finetti's theorem for Markov 
chains. Ann. Probab. 8, 1, 115-130. MR556418 

Diaconis, P. (1988). Recent progress on de Finetti's notions of exchangeabil- 
ity. In Bayesian Statistics, 3 (Valencia, 1987). Oxford Sci. Publ. Oxford Univ. 
Press, New York, 111-125. MR1008047 

Diaconis, P. and Rolles, S.W.W. (2006). Bayesian analysis for reversible 
Markov chains. The Annals of Statistics 34, no. 3. 

Durrett, R., Kesten, H., and Limic, V. (2002). Once edge-reinforced 
random walk on a tree. Probab. Theory Related Fields 122, 4, 567-592. 
MR1902191 

Horvath, L. and Shao, Q.-M. (1998). Limit distributions of directionally 
reinforced random walks. Adv. Math. 134, 2, 367-383. IMR1617789I 
Keane, M. S. (1990). Solution to problem 288. Statistica Neerlandica 44, 2, 
95-100. 

Keane, M. S. and Rolles, S. W. W. (2000). Edge-reinforced random walk 
on finite graphs. In Infinite Dimensional Stochastic Analysis (Amsterdam, 
1999). Verh. Afd. Natuurkd. 1. Reeks. K. Ned. Aka d. Wet., Vol. 52. R. Neth. 
Acad. Arts Sci., Amsterdam, 217-234. MR1832379 

Keane, M. S. and Rolles, S. W. W. (2002). Tubular recurrence. Acta 
Math. Hungar. 97, 3, 207-221. MR1933730 

Limic, V. (2003). Attracting edge property for a class of reinforced random 
walks. Ann. Probab. 31, 3, 1615-1654. MR1989445 

Maurer, S. B. (1976). Matrix generalizations of some theorems on trees, 
cycles and cocycles in graphs. SI AM J. Appl. Math. 30, 1, 143-148. MR392635 
Mauldin, R. D., Monticino, M., and von Weizsacker, H. (1996). Di- 
rectionally reinforced random walks. Adv. Math. 117, 2, 239-252 MR1371652I 
Merkl, F. and Rolles, S.W.W. (2004). Asymptotic behavior of edge- 
reinforced random walks. To appear in The Annals of Probability. 



Linearly edge- reinforced random walks 



77 



Merkl, F. and Rolles, S.W.W. (2005). Edge-reinforced random walk on 
a ladder. The Annals of Probability 33, 6, 2051-2093. 

Othmer, H. G. and Stevens, A. (1997). Aggregation, blowup, and collapse: 
the ABCs of taxis in reinforced random walks. SIAM J. Appl. Math. 57, 4, 
1044-1081. MR1462051 

Pemantle, R. (1988). Phase transition in reinforced random walk and RWRE 
on trees. Ann. Probab. 16, 3, 1229-1241. IMR942765I 

Pemantle, R. (1992). Vertex-reinforced random walk. Probab. Theory Re- 
lated Fields 92, 1, 117-136. IMRL156453I 

Pemantle, R. (2001). Random processes with reinforcement, preprint. 
Pemantle, R. and Volkov, S. (1999). Vertex-reinforced random walk on 
Z has finite range. Ann. Probab. 27, 3, 1368-1388. MR1733153 
Rolles, S. W. W. (2003). How edge-reinforced random walk arises naturally. 
Probab. Theory Related Fields 126, 2, 243-260. MR1990056 
Rolles, S.W.W. (2004). On the recurrence of edge-reinforced random walk 
on Z x G. To appear in Probability Theory and Related Fields. 
Sellke, T. (1993). Recurrence of reinforced random walk on a ladder. Tech- 
nical report 93-10, Purdue University. 

Sellke, T. (1994). Reinforced random walk on the d- dimensional integer 
lattice. Technical report 94-26, Purdue University. 

Solomon, F. (1975). Random walks in a random environment. Ann. Proba- 
bility 3, 1-31. MR362503 

Takeshima, M. (2000). Behavior of 1-dimensional reinforced random walk. 
Osaka J. Math. 37, 2, 355-372. ICTR1 7728371 

Takeshima, M. (2001). Estimates of hitting probabilities for a 1-dimensional 

reinforced random walk. Osaka J. Math. 38, 4, 693-709. MR18644IHI 

Tarres, P. (2004). Vertex-reinforced random walk on Z eventually gets stuck 

on five points. Ann. Probab. 32, 3B, 2650-2701. MR2078554 

Toth, B. (1997). Limit theorems for weakly reinforced random walks on Z. 

Studia Set. Math. Hungar. 33, 1-3, 321-337. IMR1454118I 

Vervoort, M. (2000). Games, walks and grammars. Problems I've worked 

on. PhD thesis, Universiteit van Amsterdam. 

Volkov, S. (2001). Vertex-reinforced random walk on arbitrary graphs. Ann. 
Probab. 29, 1, 66-91 . IMFU825142I 

Zabell, S. L. (1982). W. E. Johnson's "sufficientness" postulate. Ann. 
Statist. 10, 4, 1090-1099. MR673645 



